NSF Workshop on Computer Performance Evaluation

Though there are some very positive aspects of a highly detailed model, there are also some significant drawbacks. A high level of detail is good in that it permits performance models to reflect microscopic behavior. For example, a detailed model can provide measurement of the number of loads that access the data cache in any particular cycle. This ability, however, comes at the cost of being able to examine macroscopic behavior such as long-term application behavior or context switching or thread interaction; such issues can be equally important to overall performance or on the effectiveness of a new microarchitectural feature. A high level of simulator detail adds to infrastructure time, and assessing a new architectural concept can involve months of coding. In turn, this perhaps inhibits the scope of innovation. Researchers and students of architecture are less likely to think big (or different) if thinking big means putting forth a lot of effort to possibly discover nothing. As a result, we as a community are more inclined to think incremental thoughts. Finally, using highly detailed models potentially creates a false sense of accuracy in that we as a community are less likely and able to scrutinize performance results because they are justified by detailed simulation.